Integrating reinforcement learning with human demonstrations of varying ability
نویسندگان
چکیده
This work introduces Human-Agent Transfer (HAT), an algorithm that combines transfer learning, learning from demonstration and reinforcement learning to achieve rapid learning and high performance in complex domains. Using experiments in a simulated robot soccer domain, we show that human demonstrations transferred into a baseline policy for an agent and refined using reinforcement learning significantly improve both learning time and policy performance. Our evaluation compares three algorithmic approaches to incorporating demonstration rule summaries into transfer learning, and studies the impact of demonstration quality and quantity, as well as the effect of combining demonstrations from multiple teachers. Our results show that all three transfer methods lead to statistically significant improvement in performance over learning without demonstration. The best performance was achieved by combining the best demonstrations from two teachers.
منابع مشابه
Integrating Human Demonstration and Reinforcement Learning: Initial Results in Human-Agent Transfer
This work introduces Human-Agent Transfer (HAT), a method that combines transfer learning, learning from demonstration and reinforcement learning to achieve rapid learning and high performance in complex domains. Using experiments in a simulated robot soccer domain, we show that human demonstrations can be transferred into a baseline policy for an agent, and reinforcement learning can be used t...
متن کاملReward Shaping by Demonstration
Potential-based reward shaping is a theoretically sound way of incorporating prior knowledge in a reinforcement learning setting. While providing flexibility for choosing the potential function, under certain conditions this method guarantees the convergence of the final policy, regardless of the properties of the potential function. However, this flexibility of choice may cause confusion when ...
متن کاملInverse Reinforcement Learning via Ranked and Failed Demonstrations
In many robotics applications, applying reinforcement learning (RL) can be especially difficult, as it depends on the prespecification of a reward function over the environment’s states, which is often hard to define. Inverse Reinforcement Learning (IRL) [1] attempts to address this problem, by utilizing human demonstrations to learn the reward function, without having a human explicitly define...
متن کاملReinforcement Learning from Demonstration through Shaping
Reinforcement learning describes how a learning agent can achieve optimal behaviour based on interactions with its environment and reward feedback. A limiting factor in reinforcement learning as employed in artificial intelligence is the need for an often prohibitively large number of environment samples before the agent reaches a desirable level of performance. Learning from demonstration is a...
متن کاملUsing Human Demonstrations to Improve Reinforcement Learning
This work introduces Human-Agent Transfer (HAT), an algorithm that combines transfer learning, learning from demonstration and reinforcement learning to achieve rapid learning and high performance in complex domains. Using experiments in a simulated robot soccer domain, we show that human demonstrations transferred into a baseline policy for an agent and refined using reinforcement learning sig...
متن کامل